Cortical encoding and decoding models of speech production

To speak is to dynamically orchestrate the movements of the articulators (jaw, tongue, lips, and larynx), which in turn generate speech sounds. It is an amazing mental and motor feat that is controlled by the brain and is fundamental for communication. Technology that could translate brain signals into speech would be transformative for people who are unable to communicate as a result of neurological impairments. This work first investigates how articulator movements that underlie natural speech production are represented in the brain. Building upon this, this work also presents a neural decoder that can synthesize audible speech from brain signals. Data to support these results were from direct cortical recordings of the human sensorimotor cortex while participants spoke natural sentences. Neural activity at individual electrodes encoded a diversity of articulatory kinematic trajectories (AKTs), each revealing coordinated articulator movements towards specific vocal tract shapes. The neural decoder was designed to leverage the kinematic trajectories encoded in the sensorimotor cortex which enhanced performance even with limited data. In closed vocabulary tests, listeners could readily identify and transcribe speech synthesized from cortical activity. These findings advance the clinical viability of using speech neuroprosthetic technology to restore spoken communication

Parameter optimization of logistic regression classifiers

Logistic regression (LR) classifiers have been used successfully in the single-trial analysis of EEG data, especially in tasks of perceptual decision-making 12, but heuristics govern the choices for classifier parameters, such as window size (δ). Furthermore, no rigorous definition exists as to the number of epochs (N) of either class that would allow sufficient classifier training before testing using leave-one-out cross-validation. Here, we attempt to address these issues by exploring this discrete parameter space with the aid of a genetic algorithm. In doing so, we draw preliminary conclusions on both subject-specific and subject-general trends of these classifiers. To establish a baseline for comparison, we utilize EEG data from a previous study using LR to classify neural response to a two-choice forced-decision face vs. car visual task 1. In this study, a window size (δ) of 60 ms was used to segment epochs for classification. Other studies using this technique also employ a comparable window size 23, even though δ has the potential to drastically affect classifier training and performance. Similarly, the number of epochs used to train the classifier can greatly affect its performance, a number too low causing an insufficient number of points through which a dividing hyperplane can be found. Recognizing the dependence of classifier performance on these discrete parameters, we use a genetic algorithm to explore the δ vs. N design space. In doing so, we track an objective function whose value depends on maximizing an epoch window's leave-one-out A_z (area under receiver-operating characteristic) value while decreasing its variability (determined from bootstrapping), which increases with a low number of epochs. Once converging to subject-specific values of δ* and N*, we then test the classifier solution for statistical significance using the false discovery rate across all windows 4, as there are approximately E/2δ* multiple comparisons for an E milliseconds epoch with 50% window overlap. First, minimizing our objective function with N held constant at its maximum, we find that δ* can be tuned in a subject-specific way and we find on average a 3.7 ± 1.1% improvement in maximum A_z from that of the earlier study. Second, we vary δ (δ ∈ [5, 6, ..., 149, 150]ms) and N (N ∈ [10, 11, ..., N_max-1, N_max] ) simultaneously and converge using a genetic algorithm (6-bit resolution, 36-member population, 0.7 crossover probability, 0.7/(population size) mutation probability, 5) to a subject-specific δ* and N*. In each subject but one we find that N* < N_max and that δ* is a subject-specific parameter that differs from the heuristics offered by previous work. Finally, on a group level, we find that the components of our objective function exhibit distinct variation with respect to δ and N, with an epoch's maximum A_z optimizing for low N and low δ, while its A_z variability minimizes for high N and maximizes for low N, nearly irrespective of δ

Case Reports1. A Late Presentation of Loeys-Dietz Syndrome: Beware of TGFβ Receptor Mutations in Benign Joint Hypermobility

Background: Thoracic aortic aneurysms (TAA) and dissections are not uncommon causes of sudden death in young adults. Loeys-Dietz syndrome (LDS) is a rare, recently described, autosomal dominant, connective tissue disease characterized by aggressive arterial aneurysms, resulting from mutations in the transforming growth factor beta (TGFβ) receptor genes TGFBR1 and TGFBR2. Mean age at death is 26.1 years, most often due to aortic dissection. We report an unusually late presentation of LDS, diagnosed following elective surgery in a female with a long history of joint hypermobility. Methods: A 51-year-old Caucasian lady complained of chest pain and headache following a dural leak from spinal anaesthesia for an elective ankle arthroscopy. CT scan and echocardiography demonstrated a dilated aortic root and significant aortic regurgitation. MRA demonstrated aortic tortuosity, an infrarenal aortic aneurysm and aneurysms in the left renal and right internal mammary arteries. She underwent aortic root repair and aortic valve replacement. She had a background of long-standing joint pains secondary to hypermobility, easy bruising, unusual fracture susceptibility and mild bronchiectasis. She had one healthy child age 32, after which she suffered a uterine prolapse. Examination revealed mild Marfanoid features. Uvula, skin and ophthalmological examination was normal. Results: Fibrillin-1 testing for Marfan syndrome (MFS) was negative. Detection of a c.1270G > C (p.Gly424Arg) TGFBR2 mutation confirmed the diagnosis of LDS. Losartan was started for vascular protection. Conclusions: LDS is a severe inherited vasculopathy that usually presents in childhood. It is characterized by aortic root dilatation and ascending aneurysms. There is a higher risk of aortic dissection compared with MFS. Clinical features overlap with MFS and Ehlers Danlos syndrome Type IV, but differentiating dysmorphogenic features include ocular hypertelorism, bifid uvula and cleft palate. Echocardiography and MRA or CT scanning from head to pelvis is recommended to establish the extent of vascular involvement. Management involves early surgical intervention, including early valve-sparing aortic root replacement, genetic counselling and close monitoring in pregnancy. Despite being caused by loss of function mutations in either TGFβ receptor, paradoxical activation of TGFβ signalling is seen, suggesting that TGFβ antagonism may confer disease modifying effects similar to those observed in MFS. TGFβ antagonism can be achieved with angiotensin antagonists, such as Losartan, which is able to delay aortic aneurysm development in preclinical models and in patients with MFS. Our case emphasizes the importance of timely recognition of vasculopathy syndromes in patients with hypermobility and the need for early surgical intervention. It also highlights their heterogeneity and the potential for late presentation. Disclosures: The authors have declared no conflicts of interes

Effect of angiotensin-converting enzyme inhibitor and angiotensin receptor blocker initiation on organ support-free days in patients hospitalized with COVID-19

IMPORTANCE Overactivation of the renin-angiotensin system (RAS) may contribute to poor clinical outcomes in patients with COVID-19. Objective To determine whether angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) initiation improves outcomes in patients hospitalized for COVID-19. DESIGN, SETTING, AND PARTICIPANTS In an ongoing, adaptive platform randomized clinical trial, 721 critically ill and 58 non–critically ill hospitalized adults were randomized to receive an RAS inhibitor or control between March 16, 2021, and February 25, 2022, at 69 sites in 7 countries (final follow-up on June 1, 2022). INTERVENTIONS Patients were randomized to receive open-label initiation of an ACE inhibitor (n = 257), ARB (n = 248), ARB in combination with DMX-200 (a chemokine receptor-2 inhibitor; n = 10), or no RAS inhibitor (control; n = 264) for up to 10 days. MAIN OUTCOMES AND MEASURES The primary outcome was organ support–free days, a composite of hospital survival and days alive without cardiovascular or respiratory organ support through 21 days. The primary analysis was a bayesian cumulative logistic model. Odds ratios (ORs) greater than 1 represent improved outcomes. RESULTS On February 25, 2022, enrollment was discontinued due to safety concerns. Among 679 critically ill patients with available primary outcome data, the median age was 56 years and 239 participants (35.2%) were women. Median (IQR) organ support–free days among critically ill patients was 10 (–1 to 16) in the ACE inhibitor group (n = 231), 8 (–1 to 17) in the ARB group (n = 217), and 12 (0 to 17) in the control group (n = 231) (median adjusted odds ratios of 0.77 [95% bayesian credible interval, 0.58-1.06] for improvement for ACE inhibitor and 0.76 [95% credible interval, 0.56-1.05] for ARB compared with control). The posterior probabilities that ACE inhibitors and ARBs worsened organ support–free days compared with control were 94.9% and 95.4%, respectively. Hospital survival occurred in 166 of 231 critically ill participants (71.9%) in the ACE inhibitor group, 152 of 217 (70.0%) in the ARB group, and 182 of 231 (78.8%) in the control group (posterior probabilities that ACE inhibitor and ARB worsened hospital survival compared with control were 95.3% and 98.1%, respectively). CONCLUSIONS AND RELEVANCE In this trial, among critically ill adults with COVID-19, initiation of an ACE inhibitor or ARB did not improve, and likely worsened, clinical outcomes. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT0273570

Cortical encoding and decoding models of speech production

To speak is to dynamically orchestrate the movements of the articulators (jaw, tongue, lips, and larynx), which in turn generate speech sounds. It is an amazing mental and motor feat that is controlled by the brain and is fundamental for communication. Technology that could translate brain signals into speech would be transformative for people who are unable to communicate as a result of neurological impairments. This work first investigates how articulator movements that underlie natural speech production are represented in the brain. Building upon this, this work also presents a neural decoder that can synthesize audible speech from brain signals. Data to support these results were from direct cortical recordings of the human sensorimotor cortex while participants spoke natural sentences. Neural activity at individual electrodes encoded a diversity of articulatory kinematic trajectories (AKTs), each revealing coordinated articulator movements towards specific vocal tract shapes. The neural decoder was designed to leverage the kinematic trajectories encoded in the sensorimotor cortex which enhanced performance even with limited data. In closed vocabulary tests, listeners could readily identify and transcribe speech synthesized from cortical activity. These findings advance the clinical viability of using speech neuroprosthetic technology to restore spoken communication

SAMI3 data in netCDF format (2019-Mar-31)

SAMI3 (Sami3 is Also a Model of the Ionosphere) is a seamless, three-dimensional, physics-based model of the ionosphere (Huba et al, 2008). It is based on SAMI2, a two-dimensional model of the ionosphere (Huba et al., 2000). SAMI3 models the plasma and chemical evolution of seven ion species (H⁺, He⁺, N⁺, O⁺, N⁺₂, NO⁺ and O⁺₂). The temperature equation is solved for three ion species (H⁺, He⁺ and O⁺) and for the electrons. Ion inertia is included in the ion momentum equation for motion along the geomagnetic field. This is important in modeling the topside ionosphere and plasmasphere where the plasma becomes collisionless. SAMI3 includes 21 chemical reactions and radiative recombination, and uses a nonorthogonal, nonuniform, fixed grid for the magnetic latitude range +/- 89 degrees.. Drivers Neutral composition, temperature, and winds: NRLMSISE00 (Picone et al., 2002) and HWM14 (Drob et al., 2015). Solar radiation: Flare Irradiance Spectral Model version 2 (FISM v2) Magnetic field: Richmond apex model [Richmond, 1995]. Neutral wind dynamo electric field: Determined from the solution of a 2D potential equation [Huba et at., 2008]. For the SAMI3/Weimer configuration: High latitude electric field: calculated from the empirical Weimer model for the potential. For the SAMI3/AMPERE configuration: High latitude electric field: calculated using the Magnetosphere-Ionosphere Coupling solver (MIX) developed by Merkin and Lyon (2010). The inputs to MIX are SAMI3's internal conductances, plus field-aligned current observations from Active Magnetosphere and Planetary Electrodynamics Response Experiment (AMPERE), derived from the 66+ satellite Iridium NEXT constellation's engineering magnetometer data. This potential calculation is described in Chartier et al (2022). For ease of use, SAMI3 output is remapped to a regular grid using the Earth System Modeling Framework by Hill et al (2004

SAMI3 data in netCDF format (2019-Mar-30)

SAMI3 (Sami3 is Also a Model of the Ionosphere) is a seamless, three-dimensional, physics-based model of the ionosphere (Huba et al, 2008). It is based on SAMI2, a two-dimensional model of the ionosphere (Huba et al., 2000). SAMI3 models the plasma and chemical evolution of seven ion species (H⁺, He⁺, N⁺, O⁺, N⁺₂, NO⁺ and O⁺₂). The temperature equation is solved for three ion species (H⁺, He⁺ and O⁺) and for the electrons. Ion inertia is included in the ion momentum equation for motion along the geomagnetic field. This is important in modeling the topside ionosphere and plasmasphere where the plasma becomes collisionless. SAMI3 includes 21 chemical reactions and radiative recombination, and uses a nonorthogonal, nonuniform, fixed grid for the magnetic latitude range +/- 89 degrees.. Drivers Neutral composition, temperature, and winds: NRLMSISE00 (Picone et al., 2002) and HWM14 (Drob et al., 2015). Solar radiation: Flare Irradiance Spectral Model version 2 (FISM v2) Magnetic field: Richmond apex model [Richmond, 1995]. Neutral wind dynamo electric field: Determined from the solution of a 2D potential equation [Huba et at., 2008]. For the SAMI3/Weimer configuration: High latitude electric field: calculated from the empirical Weimer model for the potential. For the SAMI3/AMPERE configuration: High latitude electric field: calculated using the Magnetosphere-Ionosphere Coupling solver (MIX) developed by Merkin and Lyon (2010). The inputs to MIX are SAMI3's internal conductances, plus field-aligned current observations from Active Magnetosphere and Planetary Electrodynamics Response Experiment (AMPERE), derived from the 66+ satellite Iridium NEXT constellation's engineering magnetometer data. This potential calculation is described in Chartier et al (2022). For ease of use, SAMI3 output is remapped to a regular grid using the Earth System Modeling Framework by Hill et al (2004

Encoding of Articulatory Kinematic Trajectories in Human Speech Sensorimotor Cortex.

When speaking, we dynamically coordinate movements of our jaw, tongue, lips, and larynx. To investigate the neural mechanisms underlying articulation, we used direct cortical recordings from human sensorimotor cortex while participants spoke natural sentences that included sounds spanning the entire English phonetic inventory. We used deep neural networks to infer speakers' articulator movements from produced speech acoustics. Individual electrodes encoded a diversity of articulatory kinematic trajectories (AKTs), each revealing coordinated articulator movements toward specific vocal tract shapes. AKTs captured a wide range of movement types, yet they could be differentiated by the place of vocal tract constriction. Additionally, AKTs manifested out-and-back trajectories with harmonic oscillator dynamics. While AKTs were functionally stereotyped across different sentences, context-dependent encoding of preceding and following movements during production of the same phoneme demonstrated the cortical representation of coarticulation. Articulatory movements encoded in sensorimotor cortex give rise to the complex kinematics underlying continuous speech production. VIDEO ABSTRACT

Generalizable spelling using a speech neuroprosthesis in an individual with severe limb and vocal paralysis

Neuroprostheses have the potential to restore communication to people who cannot speak or type due to paralysis. However, it is unclear if silent attempts to speak can be used to control a communication neuroprosthesis. Here, we translated direct cortical signals in a clinical-trial participant (ClinicalTrials.gov; NCT03698149) with severe limb and vocal-tract paralysis into single letters to spell out full sentences in real time. We used deep-learning and language-modeling techniques to decode letter sequences as the participant attempted to silently spell using code words that represented the 26 English letters (e.g. "alpha" for "a"). We leveraged broad electrode coverage beyond speech-motor cortex to include supplemental control signals from hand cortex and complementary information from low- and high-frequency signal components to improve decoding accuracy. We decoded sentences using words from a 1,152-word vocabulary at a median character error rate of 6.13% and speed of 29.4 characters per minute. In offline simulations, we showed that our approach generalized to large vocabularies containing over 9,000 words (median character error rate of 8.23%). These results illustrate the clinical viability of a silently controlled speech neuroprosthesis to generate sentences from a large vocabulary through a spelling-based approach, complementing previous demonstrations of direct full-word decoding